Further Results on the Bellman Equation for Optimal Control Problems with Exit Times and Nonnegative Instantaneous Costs

We study the Bellman equation for undiscounted exit time optimal control problems for nonlinear systems and nonnegative instantaneous costs using the dynamic programming approach. Using viscosity solution theory, we prove a uniqueness theorem that characterizes the value functions for these problems as the unique viscosity solutions of the corresponding Bellman equations that satisfy appropriate boundary conditions. As a consequence, we show that the value function of ‘Fuller’s Example’ is the unique viscosity solution of the corresponding Bellman equation in the class of functions which are continuous in the plane, null at the origin, and bounded below. This extends a recent result of P. Soravia which shows that the value function for Fuller’s problem is the unique nonnegative viscosity solution of the corresponding Bellman equation which is null at the origin and continuous in the plane. Our results also apply to problems with nonconvex, nonsymmetric control sets and unbounded targets.